Filtering device and environment recognition system

ABSTRACT

A filtering device includes an evaluation value deriving module that derives, for a pair of generated images having mutual relevance, multiple evaluation values indicative of correlations between any one of blocks extracted from one of the images and multiple blocks extracted from the other image, respectively, a reference waveform part setting module that sets a reference waveform part of a transition waveform comprised of the multiple evaluation values, the reference waveform part containing the evaluation value having the highest correlation, and a difference value determining module that determines whether one or more similar waveform parts similar to the reference waveform part exist in the transition waveform, and determines, based on the result of the determination, whether the evaluation value with the highest correlation is valid as a difference value.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. 2013-205391 filed on Sep. 30, 2013, the entire contents of which arehereby incorporated by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to a filtering device and an environmentrecognition system which determine, when calculating a difference value(parallax) of an object among multiple comparison targets, whether thedifference value is valid.

2. Related Art

There are conventionally known a technique, such as collision avoidancecontrol, which detects specific objects including another vehiclelocated ahead of a vehicle, and avoids a collision with a leadingvehicle, and a technique, such as a cruise control, which controls so asto maintain an inter-vehicle distance with a leading vehicle at a safedistance (for instance, see Japanese Patent (JP-B) No. 3,349,060).

Such a collision-avoidance control and cruise control derive a parallaxby using so-called pattern matching, in order to acquire a relativedistance from the vehicle, of an object located ahead the vehicle. Thepattern matching acquires image data from, for example, each of twoimaging devices of which viewpoints differ from each other. The patternmatching then extracts any one of blocks (hereinafter, referred to as“the reference block) from an image (hereinafter, referred to as “thereference image”) based on the image data generated by one of theimaging devices, and then searches a highly-correlated block(hereinafter, referred to as “the comparison block) from an image(hereinafter, referred to as “the comparison image”) based on the imagedata generated by the other imaging device. Then, the pattern matchingrefers to imaging parameters, such as installed positions and focallengths of the imaging devices, uses so-called a stereo method or atriangulation method to calculate relative distances of the object withrespect to the imaging devices based on the derived parallax, andconverts the calculated relative distances into three-dimensional (3D)positional information which contains a horizontal distance and a heightof the object in addition to the calculated relative distances. Further,various recognition processing are performed using the 3D positionalinformation. Note that the term “horizontal” as used herein refers toscreen transverse or lateral directions, and the term “vertical(described later)” as used herein refers to screen vertical directionswhich are perpendicular to the horizontal directions.

The pattern matching calculates a correlation of a block in thecomparison image with each block in the reference image, whilehorizontally shifting the block in the comparison image, and uses adifference (difference value) in coordinates between the comparisonimage of the most-correlated block and the corresponding block in thereference image, as the parallax. However, when objects having similarfeatures are located horizontally or subject parts having similarfeatures are located horizontally in one object, one feature may bematched with other similar objects or subject parts, leading to anerroneous derivation of the parallax.

Therefore, for example, JP-B No. 3,287,465 discloses another patternmatching technique that, when one reference block in the comparisonimage has the highest correlation with multiple reference blocks in thereference image, determines only a parallax regarding the referenceblock having the minimum parallax to be valid.

However, for example, in a case where three or more similar objects orsubject parts continue, if one object in the reference image is matchedwith another object or subject part in the comparison image locatedapart from the position of the object in the reference image, thederived parallax (difference value) will be greatly different from acorrect parallax, and there may be a possibility that an object locatedaway or deeper from another object located closer to or toward thevehicle is erroneously determined to have an opposite or reversedpositional relation. In such a case, even if the technique of JP-B No.3,287,465 is adopted, still the same results will be obtained as well ifthe erroneously-derived parallax is once determined to be valid.Therefore, in such a case where similar objects or similar subject partscontinue, the parallax which should be invalidated cannot be effectivelyexcluded.

SUMMARY OF THE INVENTION

The present disclosure has been designed in consideration of thecircumstances described above, and an object thereof is to provide afiltering device and an environment recognition system, thatappropriately evaluate evaluation values of an evaluation function toeffectively exclude difference value which should be invalidated.

According to one aspect of the present disclosure, a filtering device isprovided, which includes an evaluation value deriving module thatderives, for a pair of comparison targets having mutual relevance,multiple evaluation values indicative of correlations between anextracted part that is selectively extracted from one of the comparisontargets and multiple extracted parts extracted from the other comparisontarget, respectively, a reference waveform part setting module that setsa reference waveform part of a transition waveform comprised of themultiple evaluation values, the reference waveform part containing theevaluation value with the highest correlation, and a difference valuedetermining module that determines whether one or more similar waveformparts similar to the reference waveform part exist in the transitionwaveform, and determines, based on the result of the determination,whether the evaluation value with the highest correlation is valid as adifference value.

The difference value determining module may select as the similarwaveform part, a waveform part of which one or more parameters are closeto the parameters of the reference waveform part. The one or moreparameters may be selected from the group consisting of: an absolutevalue of the evaluation value at which an inclination of the transitionwaveform becomes zero; a difference between the evaluation value withthe highest correlation and an evaluation value of an inflection pointat which the inclination of the transition waveform becomes zero for thefirst time; and a horizontal difference between the evaluation valuewith the highest correlation and an evaluation value of an inflectionpoint that satisfies a predetermined condition.

The reference waveform part setting module may select as the referencewaveform part either one of two waveform parts located from a firstinflection point at which the evaluation value has the highestcorrelation to two, second and third inflection points before and afterthe first inflection point in horizontal directions at whichinclinations of the transition waveform become zero for the first time,respectively, the either one of the waveform parts having a largerdifference between the evaluation value of the first inflection pointwith the highest correlation and an evaluation value of either one ofthe second and third inflection points.

The comparison target may be an image and the extracted part may be ablock consisting of one or more pixels in the image.

According to another aspect of the present disclosure, an environmentrecognition system is provided, which includes one or more imagingdevices that generate a pair of images having mutual relevance, anevaluation value deriving module that derives, for the pair of generatedimages, multiple evaluation values indicative of correlations betweenany one of blocks extracted from one of the images and multiple blocksextracted from the other image, respectively, a reference waveform partsetting module that sets a reference waveform part of a transitionwaveform comprised of the multiple evaluation values, the referencewaveform part containing the evaluation value having the highestcorrelation, and a difference value determining module that determineswhether one or more similar waveform parts similar to the referencewaveform part exist in the transition waveform, and determines, based onthe result of the determination, whether the evaluation value with thehighest correlation is valid as a parallax.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not by wayof limitation in the figures of the accompanying drawings, in which thelike reference numerals indicate like elements and in which:

FIG. 1 is a block diagram illustrating a connecting relation of anenvironment recognition system;

FIG. 2 is a functional block diagram schematically illustratingfunctions of a vehicle exterior environment recognition device;

FIGS. 3A to 3C illustrate average value difference matching;

FIG. 4 is illustrates a reference image and a comparison image;

FIGS. 5A and 5B are graphs illustrating transitions of evaluation valuesin two arbitrary areas;

FIGS. 6A and 6B are diagrams illustrating particular processing ofinflection points;

FIG. 7 is a graph illustrating a reference waveform part;

FIGS. 8A and 8B are graphs illustrating narrow-downs of similar waveformparts by a difference value determining module;

FIG. 9 is a graph illustrating the narrow-down of similar waveform partsby the difference value determining module; and

FIG. 10 is a graph illustrating the narrow-down of similar waveformparts by the difference value determining module.

DETAILED DESCRIPTION

Hereinafter, a suitable example of the present disclosure will bedescribed in detail with reference to the accompanying drawings.Dimensions, material, concrete numerical values, etc. illustrated inthis example are merely instances for easier understanding of thepresent disclosure, and, unless otherwise particularly specified, theseinstances are not intended to limit the present disclosure. Note that,in this description and the accompanying drawings, elements havingsubstantially the same function and configuration are denoted with thesame reference numerals in order to omit redundant explanations, and anyother elements which are not directly related to the present disclosureare not illustrated in the accompanying drawings.

In recent years, vehicles having so-called a collision avoidancefunction (adaptive cruise control: ACC) have been spreading. Thisfunction images the road environment ahead of a vehicle where on-boardcameras are mounted, identifies an object, such as leading vehicle,based on color information and/or positional information obtained fromthe images (comparison targets), and thereby avoiding a collision withthe identified object and/or maintains an inter-vehicle distance withthe leading vehicle at a safe distance.

The collision avoidance function uses, for example, a pair of imagingdevices of which viewpoints differ from each other in order to acquire arelative distance from the vehicle, of objects located ahead of thevehicle, compares an image acquired from one of the imaging devices withanother image acquired from the other imaging device to extracthighly-correlated blocks (extracted parts) by using so-called patternmatching. These images are used herein as a pair of comparison targets,and one is called a “reference image,” and the other is called a“comparison image.” However, if multiple objects have similar featurescontinuously appearing in the horizontal directions in the images, or ifone object has multiple subject parts having similar features appearingcontinuously in the horizontal directions, the object or the subjectpart may be matched with another similar object or subject parts,resulting in an erroneous derivation of a parallax (a difference value).Thus, a purpose of this example is to determine whether a transitionwaveform that is a waveform formed by an evaluation value in anevaluation function of the pattern matching, that is, a waveform that issimilar to a waveform near a minimum of the evaluation value (a valuewhere correlation becomes highest) exists (i.e., whether the waveformhas repeatability), and then effectively exclude a parallax which shouldbe invalidated. Note that the term “block” as used herein refers to partof an image comprised of one or more pixels. Below, an environmentrecognition system for achieving such a purpose will be described, aswell as a filtering device provided to a vehicle exterior environmentrecognition device that is a particular component of the system will bedescribed in detail.

(Environment Recognition System 100)

FIG. 1 is a block diagram illustrating a connecting relation of theenvironment recognition system 100. The environment recognition system100 is comprised of a pair of imaging devices 110 mounted inside avehicle 1 (hereinafter, simply referred to as “the vehicle”), a vehicleexterior environment recognition device 120, and a vehicle controldevice 130 (which is typically comprised of an electronic control unit(ECU)).

Each imaging device 110 is comprised of imaging elements, such ascharge-coupled devices (CCDs) or complementary metal-oxidesemiconductors (CMOSs). Each imaging device 110 can image theenvironment ahead of the vehicle 1 to generate a color image consistingof three hues (R (red), G (green) and B (blue)) or a monochrome image.Note that the color image imaged by the imaging device 110 is adopted asa luminance image to distinguished it from a distance image describedlater.

Moreover, the two imaging devices 110 are disposed so as to be separatedfrom each other in substantially lateral or horizontal directions suchthat they are oriented facing to the traveling direction of the vehicle1 to have their optical axes being oriented substantially parallel toeach other. Each imaging device 110 sequentially generates image datawhich is obtained by imaging objects existing within a detection areaahead of the vehicle 1 per frame, for example, at every 1/60 seconds(i.e., 60 fps). Note that the term “object” to be recognized as usedherein refers not only to a solid object existing independently, such asa vehicle, a pedestrian, a traffic light, a road surface (travelingpath), a guardrail, and a building, but also to an object that can beidentified as part of the solid object, such as a taillight, a blinker,each illuminating part of the traffic light. Each functional moduledescribed below carries out processing for every frame, triggered by arefreshing of such image data.

The vehicle exterior environment recognition device 120 acquires theimage data from each of the two imaging devices 110, derives theparallax using so-called pattern matching, and associates the derivedparallax information (corresponding to a relative distance describedlater) with the image data to generate the distance image. The patternmatching will be described later in detail. Further, the vehicleexterior environment recognition device 120 uses the luminance based onthe luminance image and three-dimensional (3D) positional information inreal space containing the relative distance with respect to the vehicle1 based on the distance image to group blocks, of which the luminanceare equal and the 3D positional information are close to each other, asone unitary object, and then identifies specific objects to which theobject in the detection area ahead of the vehicle 1 corresponds.

When the specific object is identified, the vehicle exterior environmentrecognition device 120 derives a relative speed and the like of thespecific object (for example, a leading vehicle) while tracking thespecific object, and then determines whether a possibility of thespecific object colliding with the vehicle 1 is high. Here, if thevehicle exterior environment recognition device 120 determines that thepossibility of a collision is high, the vehicle exterior environmentrecognition device 120 then gives (informs) a vehicle operator a warningindication through a display unit 122 installed in front of the operatorand outputs information indicative of the warning to the vehicle controldevice 130.

The vehicle control device 130 accepts operational inputs of theoperator through a steering wheel 132, an accelerator (gas pedal) 134,and a brake pedal 136, and transmits the inputs to a steering mechanism142, a drive mechanism 144, and a brake mechanism 146, respectively, tocontrol the vehicle 1. The vehicle control device 130 also controls thedrive mechanism 144 and the brake mechanism 146 according toinstructions from the vehicle exterior environment recognition device120.

Next, the configuration of the vehicle exterior environment recognitiondevice 120 will be described in detail. Note that only processing forobtaining the parallax of the object by the filtering device, which is afeature of this example, will be described in detail herein and, thus,description of any other configurations unrelated to the feature of thisexample will be omitted.

(Vehicle Exterior Environment Recognition Device 120)

FIG. 2 is a functional block diagram schematically illustratingfunctions of the vehicle exterior environment recognition device 120. Asillustrated in FIG. 2, the vehicle exterior environment recognitiondevice 120 is comprised of an I/F unit 150, a data holding unit 152, anda central controlling unit 154.

The I/F unit 150 is an interface that performs bidirectional informationexchanges with the imaging devices 110 and the vehicle control device130. The data holding unit 152 is comprised of one or more RAMs, one ormore flash memories, one or more HDDs, etc. and holds variousinformation required for the processing of each functional moduledescribed below and temporarily holds the image data received from theimaging devices 110.

The central controlling unit 154 is comprised of one or more integratedcircuits containing one or more central processing units (CPUs), one ormore ROMs where one or more programs and the like are stored, or one ormore RAMs as work areas, and controls the I/F unit 150, the data holdingunit 152, etc. through a system bus 156. In this example, the centralcontrolling unit 154 also functions as an evaluation value derivingmodule 160, a reference waveform part setting module 162, and adifference value determining module 164. The evaluation value derivingmodule 160, the reference waveform part setting module 162, and thedifference value determining module 164 also function as the filteringdevice. Next, the pattern matching processing will be described indetail.

(Pattern Matching Processing)

The evaluation value deriving module 160 acquires the image data fromeach of the two imaging devices 110. Based on the acquired two pieces ofimage data which have mutual relevance, the evaluation value derivingmodule 160 then extracts any one of blocks from one of the images whichwill be used as a reference (reference image), while extracting multipleblocks (comparison targets) from the other image (comparison image). Theevaluation value deriving module 160 derives multiple evaluation valuesindicative of correlations between the reference block and thecomparison block.

The pattern matching may be a comparison in luminance (Ycolor-difference signal) per block between the pair of images. Forexample, the pattern matching includes approaches, such as a sum ofabsolute difference (SAD) in which a difference in luminance iscalculated, a sum of squared intensity difference (SSD) which usesvalues obtained by squaring the differences, and a normalized crosscorrelation (NCC) which uses similarities of variances obtained bysubtracting an average value of the luminance of pixels from theluminance of each pixel. Among these approaches, SAD will beparticularly described herein as an instance. In this example, averagevalue difference matching is also performed. This average valuedifference matching calculates an average value of the luminance ofpixels around a block in the reference image and the comparison image,respectively, and subtracts each average value from the luminance of thepixels within the block to derive evaluation values. Next, the averagevalue difference matching will be described in detail.

FIGS. 3A to 3C illustrate the average value difference matching. Asillustrated in FIG. 3A, the evaluation value deriving module 160 firstextracts a block 204 (hereinafter, referred to as “the reference block”)comprised of a matrix of pixels 202, for example, consisting of 4 pixelsin the horizontal directions×4 pixels in the vertical directions, fromthe reference image 200. The evaluation value deriving module 160sequentially repeats the processing per block, by extracting anotherreference block 204 and deriving the parallax for each extractedreference block 204. While the reference block 204 is comprised of 4pixels in the horizontal directions×4 pixels in the vertical directionsin this example, any number of pixels within the reference block 204 maybe selected.

One reference block 204 is extracted so as not to overlap with anotheradjacent reference block 204. In this example, the adjacent referenceblocks 204 are extracted, and thus all 6,750 blocks (150 blocks in thehorizontal directions×45 blocks in the vertical directions) aresequentially extracted as the reference block 204, for all the pixels202 displayed within the detection area (for example, 600 pixels in thehorizontal directions×180 pixels in the vertical directions).

Since the average value difference matching is adopted in this exampleas described above, the evaluation value deriving module 160 calculates,as illustrated in FIG. 3B, an average value Ab of luminance Rb(i, j) ofthe pixels 202 within an area 206 represented by 8 pixels in thehorizontal directions×8 pixels in the vertical directions around thereference block 204 centering on the reference block 204 based on thefollowing Equation 1:

Ab=ΣRb(i,j)/64  (Equation 1)

Note that “i” is a horizontal pixel position in the area 206 (i=1 to 8),and “j” is a vertical pixel position in the area 206 (j=1 to 8). If thearea 206 is partially located outside the reference image 200 (i.e., thearea 206 is partially missing at an end of the reference image 200), theaverage value Ab is calculated while the missing part is omitted.

The evaluation value deriving module 160 subtracts the above-describedaverage value Ab from a luminance Eb(i, j) of each of the pixels 202within the reference block 204 to derive an average value differenceluminance EEb(i, j) as the following Equation 2;

EEb(i,j)=Eb(i,j)−Ab  (Equation 2)

Note that “i” is a horizontal pixel position in the reference block 204(i=1 to 4), and “j” is a vertical pixel position in the reference block204 (j=1 to 4).

The evaluation value deriving module 160 then extracts a block 214(hereinafter, referred to as “the comparison block”) represented by thematrix of pixels 212, for example, of 4 pixels in the horizontaldirections×4 pixels in the vertical directions from the comparison image210, as illustrated in FIG. 3C. Note that the evaluation value derivingmodule 160 sequentially extracts, for each one of the reference blocks204, multiple comparison blocks 214, and derives the evaluation valuesindicative of correlations with the respective reference blocks 204.

The comparison block 214 is shifted by, for example, 1 pixel at a timein the horizontal direction and then extracted and, thus, the pixels ofthe adjacent comparison blocks 214 are overlapped. In this example, thetotal of 128 comparison blocks 214 to the left and right in thehorizontal direction are extracted, for each one of the reference blocks204, with respect to a position 216 corresponding to the reference block204. Therefore, the extraction area (search area) has 131 pixels (=128+3pixels) in the horizontal directions×4 pixels in the verticaldirections. The positional relationship between the position 216corresponding to the reference block 204 and the extraction area is setaccording to the appearing pattern of the parallaxes between thereference image 200 and the comparison image 210.

Since the average value difference matching is adopted in this exampleas described above, the evaluation value deriving module 160 calculatesan average value Ac of luminance Rc(i, j) of pixels within an arearepresented by 8 pixels in the horizontal directions×8 pixels in thevertical directions around the comparison block 214 centering on thecomparison block 214 based on the following Equation 3, similar to thereference block 204:

Ac=ΣRc(i,j)/64  (Equation 3)

Note that “i” is a horizontal pixel position in the area (i=1 to 8), and“j” is a vertical pixel position in the area (j=1 to 8).

Next, the evaluation value deriving module 160 subtracts theabove-described average value Ac from a luminance Ec(i, j) of each ofthe pixels within the comparison block 214 to derive an average valuedifference luminance EEc(i, j), as the following Equation 4:

EEc(i,j)=Ec(i,j)−Ac  Equation 4)

Note that “i” is a horizontal pixel position in the comparison block 214(i=1 to 4), and “j” is a vertical pixel position in the comparison block214 (j=1 to 4).

Next, the evaluation value deriving module 160 subtracts, from theaverage value difference luminance EEb(i, j) of each pixel 202 of thereference block 204, the average value difference luminance EEc(i, j) ofeach pixel 212 corresponding to the same position in the comparisonblock 214, and integrates the subtraction results to derive anevaluation value S, as illustrated in the following Equation 5:

S=Σ(EEb(i,j)−EEc(i,j))  (Equation 5)

Thus, the multiple derived evaluation values S have higher correlationsas the evaluation values S themselves have smaller values, i.e., smallerdifferences. Therefore, among the multiple evaluation values (here, 128evaluation values) of one reference block 204 with the comparison blocks214, the position of the minimum evaluation value (minimum value) servesas a candidate of the position indicating an end of the parallaxes.

When the minimum evaluation value for any reference block 204 isderived, the evaluation value deriving module 160 holds, in apredetermined area of the data holding unit 152, a difference betweencoordinates of the comparison block 214 corresponding to the minimumvalue and coordinates of the reference block 204 (the position 216corresponding to the reference block 204) as the parallax. Therefore,the data holding unit 152 holds the parallaxes of 6,750 reference blocks204 (=150 blocks in the horizontal directions×45 blocks in the verticaldirections).

The above-described average value difference matching only useshigh-frequency components of the image for the matching, and can removelow-frequency noise because it has an equivalent function to a high-passfilter. In addition, the matching has high accuracy of identifying theparallaxes and thus can improve accuracy of deriving the parallaxes,even under the effects of slight imbalance in luminance between thereference image 200 and the comparison image 210, and the effects ofgain variation due to aging of the cameras (imaging devices) and/oranalog circuit components.

FIG. 4 is illustrates the reference image 200 and the comparison image210. The upper part (a) of FIG. 4 illustrates the image from one of theimaging devices 110 located on the right, while the lower part (b)illustrates the corresponding image from the other imaging device 110located on the left. In this example, the right image is used as thereference image 200, and the left image is used as the comparison image210.

When the evaluation value deriving module 160 derives the evaluationvalues based on the reference image 200 and the comparison image 210, itis easy to derive the parallaxes and perform the pattern matchingbecause an edge appears at a side end of a track in an area 220illustrated in the upper part (a) of FIG. 4.

On the other hand, like a nose barrier in an area 222 illustrated in theupper part (a) of FIG. 4, subject parts having similar features (here, apattern of fence) are repeated continuously in the horizontaldirections, the reference block 204 of the reference image 200 ismatched with the comparison block 214 which should not be matched withinthe comparison image 210, leading to erroneous derivations of theparallaxes.

FIGS. 5A and 5B are graphs illustrating transitions of the evaluationvalues in the two areas 220 and 222 described above. FIG. 5A illustratesan instance of any one of the reference blocks 204 in the area 220, andFIG. 5B illustrates an instance of any on of the reference blocks 204 inthe area 222. In FIGS. 5A and 5B, the horizontal axis represents thehorizontal position of the comparison block 214, and the vertical axisrepresents the evaluation value. Smaller evaluation values indicatehigher correlations.

In FIG. 5A, since the blocks compared in the area 220 arecharacteristic, the evaluation value projects locally toward a smallervalue, and a minimum value (i.e., global minima) appears clearly. On theother hand, in FIG. 5B, multiple inflection points (points at whichinclinations become zero) appear corresponding to minimum values (i.e.,local minimum) of the evaluation value where the waveforms becomedownwardly convexes. Thus, depending on display modes of the referenceimage 200 and the comparison image 210, another inflection point (orlocal minima) located near the global minima may be erroneouslyrecognized as the global minima, leading to an erroneous recognition ofthe parallax at the incorrect inflection point.

For this reason, the vehicle exterior environment recognition device 120of this example focuses on the transition waveform which is a waveformformed by the evaluation values of the pattern matching. The vehicleexterior environment recognition device 120 determines whether anotherwaveform part similar to a waveform part near the global minima of theevaluation value (where the correlation becomes the highest in theentire transition waveform) exists, that is, whether a transitionwaveform has repeatability. If multiple similar waveform parts exist inthe transition waveform, the vehicle exterior environment recognitiondevice 120 determines that the evaluation value is not reliable and isinvalidated as the parallax. Below, this processing will be described inmore detail.

The reference waveform part setting module 162 identifies the inflectionpoint based on the transition waveform comprised of the multipleevaluation values, and sets a reference waveform part containing theglobal minima based on the evaluation value of which the correlation ishighest and a inflection point selected from the identified inflectionpoints.

FIGS. 6A and 6B are diagrams illustrating particular processing of theinflection points. Since the inflection point is a point at which theinclination of the transition waveform becomes zero, both of two pointsbefore and after the inflection point in the horizontal directions(i.e., one point before the inflection point and one point after theinflection point) are equal to or greater than or less than theinflection point. Specifically, suppose that the horizontal positionsand the evaluation values of the comparison block 214 at three adjacentpoints are (x−1, s0), (x, s1), and (x+1, s2), respectively. If therelation of these points satisfies the Equation 6 or 7 described below,the relation of the points becomes as illustrated in FIG. 6A, and theevaluation value s1 of the horizontal position x is at thedownwardly-convex inflection point (hereinafter, referred to as “thetrough inflection point”).

s0≧s1 and s1<s2  (Equation 6)

s0>s1 and s1≦s2  (Equation 7)

On the other hand, if the relation of the points satisfies the Equation8 or 9 described below, the relation of the points becomes asillustrated in FIG. 6B, and the evaluation value s1 of the horizontalposition x is at the upwardly-convex inflection point (hereinafter,referred to as “the crest inflection point”).

s0≦s1 and s1>s2  (Equation 8)

s0<s1 and s1≧s2  (Equation 9)

FIG. 7 is a graph illustrating the reference waveform part. Thereference waveform part setting module 162 derives differences betweenthe minimum value which is also the trough inflection point and theevaluation values at the two inflection points (crest inflection points)which are adjacent before and after in the horizontal directions to theminimum value (where the inclination of the transition waveform becomeszero for the first time), respectively. Hereinafter, the difference inthe evaluation value from the crest inflection point located at the leftside in the horizontal direction is referred to as “difference Hl”, andthe difference in the evaluation value from the crest inflection pointlocated at the right side in the horizontal direction is referred to as“difference Hr”. Then, as illustrated in FIG. 7, the reference waveformpart setting module 162 sets, as the reference waveform part, either oneof the two waveform parts, whichever has a greater difference in theevaluation value between the minimum value and the crest inflectionpoint (here, a transition waveform up to the inflection point on thedifference Hl side in the evaluation value. Of course, if the differenceHr in the evaluation value is greater than the difference Hl, thetransition waveform up to the inflection point on the difference Hr sidein the evaluation value is selected as the reference waveform part.

As described above, since the reference waveform part is formed by awaveform part only with one of the inflection points with respect to theminimum value, processing load at the time of comparing the waveformparts can be reduced. In addition, it is possible to extract waveformparts similar to the reference waveform part with high precision byselecting, as the inflection point, an inflection point having a greaterdifference (i.e., more characteristic) in the evaluation value from theminimum value.

The difference value determining module 164 determines whether similarwaveform parts which are similar to the reference waveform part exist inthe transition waveform, and, based on the results, it determineswhether the minimum evaluation value is valid as the parallax. Thedifference value determining module 164 narrows down the waveformthrough multiple conditions in order to determine whether the similarwaveform parts which are similar to the reference waveform part exist todetect waveform part(s) which satisfies all the conditions. In thisexample, the waveforms which satisfy three conditions are selected asthe similar waveform parts.

FIGS. 8A and 8B, and 9 and 10 are graphs illustrating the narrow-downsof the similar waveform parts by the difference value determining module164. As illustrated in FIG. 8A, the difference value determining module164 first extracts from the transition waveform as candidates of thesimilar waveform parts illustrated by solid lines in FIG. 8A, all thewaveforms which have inclinations equivalent to the reference waveformpart illustrated by a dashed dotted line, that is, waveforms each havingthe crest inflection point on the left side in the horizontal directionand the trough inflection point on the right side in the horizontaldirection. Next, as illustrated in FIG. 8B, the difference valuedetermining module 164 eliminates waveform parts illustrated by dashedlines in this drawing, of which absolute values of the evaluation valuesat the trough inflection points are not close to the trough inflectionpoint (minimum value) of the reference waveform part, that is, of whichthe absolute values of the evaluation values at the trough inflectionpoints are equal to or greater than a predetermined threshold. Thus,only waveforms of which the absolute values of the evaluation values atthe trough inflection points are less than the predetermined thresholdwill remain as the candidates of the similar waveform parts.

Note that the predetermined threshold D is determined by the followingEquation 10:

D=H×Gd+P+B  (Equation 10)

Note that H is difference Hl or Hr in the evaluation value describedabove, whichever is greater than the other, Gd is a gain which is set,and P is the minimum value, and B is an optionally-set offset.

According to the above configuration, the waveform parts of which theabsolute values of the evaluation values at the trough inflection pointsare apart from the minimum value, that is, which should not be acceptedas the parallaxes, can be effectively eliminated.

Next, as illustrated in FIG. 9, the difference value determining module164 eliminates, from the waveform parts narrowed down by Equation 10,waveform parts illustrated by dashed lines which are not close to thedifference in the evaluation value between the minimum value (localminima) and the crest inflection point (the inflection point at whichthe inclination of the transition waveform becomes zero for the firsttime), that is, differences V in the evaluation value between the crestinflection point and the trough inflection point are not within apredetermined range. Thus, only waveform parts of which the differencesV in the evaluation value between the crest inflection point and thetrough inflection point are within the predetermined range will remainas the candidates of the similar waveform parts.

Note that whether the difference V is within the predetermined range isdefined by the following Equation 11:

H×Gl<V<H×Gh  (Equation 11)

Note that H is the difference Hl or Hr in the evaluation value describedabove, whichever is greater than the other, and Gl and Gh are gainswhich define an upper limit and a lower limit, respectively.

According to this configuration, the waveform parts which should not bedetermined to be similar to the reference waveform part, of which thedifferences V in the evaluation value between the crest inflectionpoints and the trough inflection points are apart from the difference inthe evaluation value between the crest inflection point and the troughinflection point in the reference waveform part, can be effectivelyeliminated.

Next, as illustrated in FIG. 10, the difference value determining module164 eliminates from the waveform parts narrowed down by Equation 11,waveform parts (only one is applicable in FIG. 10) illustrated by adashed line which is not close to a horizontal difference from theminimum value (global minima) to a trough inflection point whichsatisfied the predetermined conditions, that is, as illustrated in theEquation 12 described below, waveform part(s) of which an absolute valueof a difference between a horizontal difference W from the minimum value(global minima) to a predetermined trough inflection point and ahorizontal difference X between the trough inflection points of thewaveform parts narrowed down by Equation 11, are equal to or greaterthan a predetermined threshold T. Thus, only waveform parts of which thehorizontal differences between the trough inflection points are close tothe horizontal difference from the minimum value (global minima) to thepredetermined trough inflection point will remain as the candidates ofthe similar waveform parts.

|W−X|<T  (Equation 12)

Note that the predetermined trough inflection point is a troughinflection point of one of two waveform parts which are located beforeand after the reference waveform part in the horizontal directions,among the waveform parts narrowed down by Equation 11. The troughinflection point of the waveform part located at the left side of thereference waveform part is used as the predetermined trough inflectionpoint in the instance of FIG. 10. Aleternatively, the trough inflectionpoint of the waveform part located at the right side of the referencewaveform part may also be used as the predetermined trough inflectionpoint. The difference value determining module 164 selects the waveformsthus remained as the similar waveform parts.

According to this configuration, the waveform parts of which thehorizontal difference X between the trough inflection points issignificantly different from the horizontal difference W from theminimum value (global minima) to the predetermined trough inflectionpoint, and the waveform parts of which the distance between the waveformparts should not be accepted to be similar to the distance between thewaveform parts containing the reference waveform part, can beeffectively eliminated.

Next, the difference value determining module 164 determines whether theminimum value is valid based on the one or more similar waveform partswhich are waveform parts narrowed down by Equation 12, as the parallax.Specifically, based on whether the number of similar waveform parts (orthe number of trough inflection points thereof) equals to or greaterthan a predetermined threshold (e.g., 2 or more), the difference valuedetermining module 164 invalidates the minimum value as the parallax ifthe number of similar waveform parts equals to or greater than thepredetermined threshold as illustrated in FIG. 10.

Alternatively or additionally to the number of the similar waveformparts described above, the similar waveform part may be determined to belocated before and after the reference waveform part in the horizontaldirections. For example, in the instance of FIG. 10, since the similarwaveform parts exist both before and after the reference waveform partin the horizontal directions, the minimum value is invalidated as theparallax.

As described above, in this example, it is determined, through multipleconditions, whether waveform part(s) similar to a waveform part near theminimum evaluation value exists, that is, whether the transitionwaveform has repeatability. If multiple similar waveform parts exist inthe transition waveform, the minimum value is determined to beunreliable and the minimum value is invalidated as the parallax. Thus,it is possible to effectively exclude the parallaxes which should beinvalidated.

One or more programs which cause one or more computers to function asthe filtering device and/or environment recognition system describedabove, and one or more storage media which record the program(s), suchas flexible discs, magneto-optic discs, ROMs, CDs, DVDs, BDs, which canbe read by the computer(s), are also provided. Note that the term“program” as used herein refers to a data set described with anylanguages and any describing methods.

As above, although the suitable example of the present disclosure isdescribed with reference to the accompanying drawings, the presentdisclosure is not intended to be limited to the above example. It isapparent that a person skilled in the art can reach various changes ormodifications without departing from the scope of the present disclosuredescribed in the appended claims, and it should be understood that thosederivatives naturally encompass the technical scope of the presentdisclosure.

For example, in the above example, although the pair of images which aresimultaneously imaged by the two imaging devices 110 of which viewpointsdiffer from each other are used as the comparison targets, the presentdisclosure is widely applicable to a pair of images which have mutualrelevance, without limiting to the images described above. Such a pairof images having mutual relevance include, for example, two images whichare imaged by a single imaging device (e.g., a monocular camera) atdifferent timings and outputted in a time series, which are processingtargets of so-called an optical flow; and a combination of an imagecaptured or imaged and an image prepared in advance, which areprocessing targets of so-called template matching. Although the“parallax” between the pair of images simultaneously imaged by the twoimaging devices 110 of which viewpoints differ from each other isdescribed as the difference value in the example described above, thedifference value may be a difference between corresponding extractedparts, such as a difference between reference blocks in the pair ofimages having mutual relevance, without limiting to the above.

Further, although the luminance images are used as the comparisontargets to derive the evaluation values based on the luminances of theluminance images in the example described above, the evaluation valuesmay also be derived based on information other than the luminances, suchas a heat distribution acquired from a far-infrared camera, or adistribution of reflection intensity obtained from a laser radar or amillimeter wave radar, as the comparison target. Also in this case, adifference value refers to the difference between the correspondingextracted parts, similar to the above.

The present disclosure can be used for the filtering device and theenvironment recognition system which determine, when calculating thedifference value (parallax) of the object in the multiple images,whether the difference value is valid.

1. A filtering device, comprising: an evaluation value deriving modulethat derives, for a pair of comparison targets having mutual relevance,multiple evaluation values indicative of correlations between anextracted part that is selectively extracted from one of the comparisontargets and multiple extracted parts extracted from the other comparisontarget, respectively; a reference waveform part setting module that setsa reference waveform part of a transition waveform comprised of themultiple evaluation values, the reference waveform part containing theevaluation value with the highest correlation; and a difference valuedetermining module that determines whether one or more similar waveformparts similar to the reference waveform part exist in the transitionwaveform, and determines, based on the result of the determination,whether the evaluation value with the highest correlation is valid as adifference value.
 2. The filtering device of claim 1, wherein thedifference value determining module selects as the similar waveformpart, a waveform part of which one or more parameters are close to theparameters of the reference waveform part, the one or more parametersbeing selected from the group consisting of: an absolute value of theevaluation value of an inflection point at which an inclination of thetransition waveform becomes zero; a difference between the evaluationvalue with the highest correlation and an evaluation value of aninflection point at which the inclination of the transition waveformbecomes zero for the first time; and a horizontal difference between theevaluation value with the highest correlation and an evaluation value ofan inflection point that satisfies a predetermined condition.
 3. Thefiltering device of claim 1, wherein the reference waveform part settingmodule selects as the reference waveform part either one of two waveformparts located from a first inflection point at which the evaluationvalue has the highest correlation to two, second and third inflectionpoints before and after the first inflection point in horizontaldirections at which inclinations of the transition waveform become zerofor the first time, respectively, the either one of the waveform partshaving a larger difference between the evaluation value of the firstinflection point with the highest correlation and an evaluation value ofeither one of the second and third inflection points.
 4. The filteringdevice of claim 2, wherein the reference waveform part setting moduleselects as the reference waveform part either one of two waveform partslocated from a first inflection point at which the evaluation value hasthe highest correlation to two, second and third inflection pointsbefore and after the first inflection point in horizontal directions atwhich inclinations of the transition waveform become zero for the firsttime, respectively, the either one of the waveform parts having a largerdifference between the evaluation value of the first inflection pointwith the highest correlation and an evaluation value of either one ofthe second and third inflection points.
 5. The filtering device of claim1, wherein the comparison target is an image and the extracted part is ablock consisting of one or more pixels in the image.
 6. The filteringdevice of claim 2, wherein the comparison target is an image and theextracted part is a block consisting of one or more pixels in the image.7. The filtering device of claim 3, wherein the comparison target is animage and the extracted part is a block consisting of one or more pixelsin the image.
 8. The filtering device of claim 4, wherein the comparisontarget is an image and the extracted part is a block consisting of oneor more pixels in the image.
 9. An environment recognition system,comprising: one or more imaging devices that generate a pair of imageshaving mutual relevance; an evaluation value deriving module thatderives, for the pair of generated images, multiple evaluation valuesindicative of correlations between any one of blocks extracted from oneof the images and multiple blocks extracted from the other image,respectively; a reference waveform part setting module that sets areference waveform part of a transition waveform comprised of themultiple evaluation values, the reference waveform part containing theevaluation value having the highest correlation; and a difference valuedetermining module that determines whether one or more similar waveformparts similar to the reference waveform part exist in the transitionwaveform, and determines, based on the result of the determination,whether the evaluation value with the highest correlation is valid as aparallax.